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CLAIMS 

What is claimed is: 

1. A vision-based identification apparatus comprising: 
a host vehicle; 

wherein said host vehicle is equipped with a plurality of elements including: 

i) an image capture device element; operatively interconnected with; 

ii) an image signal processor element; which is operatively interconnected with; 

iii) a matching processor element; which is also operatively interconnected with; 

iv) a radar transceiver element; 

wherein the image capture device is configured to provide a time based sequence of data 
fi-ames to the image signal processor and the image signal processor provides a processed 
image signal to the matching processor, and 

wherein the data firames include a two dimensional array of pixel elements; and 
wherein the radar transceiver is configured to provide a radar signal to the matching 
processor, and 

wherein the matching processor combines the processed image signal and the radar 
signal, whereby the combined signals complement each other and allow the apparatus to 
effectively identify objects likely to be misidentified as collision threats. 

2. The vision-based identification apparatus of claim 1, wherein the image capture 
device is a video camera responsive to electromagnetic radiation substantially in at least 
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one of the regions selected from the group consisting of the visible region, and the 
infrared region. 

3. The vision-based identification apparatus of claim 1, wherein the time based sequence 
of data frames include a plurality of data elements including at least one horizontal edge; 
and 

wherein each horizontal edge is identified based on a plurality of pixels having a 
substantially similar electromagnetic radiation response across a plurality of substantially 
horizontally aligned pixels; and 

wherein the image signal processor extracts horizontal edges, from the time based 
sequence of data frames, in the form of edge pixels, and 

wherein the edge pixels are projected in each row of the data frames, to get a horizontal 
edge projection in the sequence of data frames; and 

wherein each horizontal edge projection may be tracked in time based on the horizontal 
edge projection's sequential appearance in the data frames. 

4. The vision-based identification apparatus of claim 3, wherein horizontal edge 
projections may be tracked in time by recording sequential data frames and matching 
horizontal edge projections in the sequential data frames while allowing, in the sequential 
data frames, for minor variations from, in at least one of the following categories: 

i. the relative position of the horizontal edge projection in the frame; 

ii. the relative orientation of the horizontal edge projection in the frame; and 

iii. the relative length of the horizontal edge projection; and 
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wherein at least one new tracking sequence can exist for horizontal edge projections that 
have predefined characteristics but were not present in prior data frames. 

5. The vision-based identification apparatus of claim 4, wherein allowed, minor 
5 variations in the sequential data frames provide data that allows for the determination and 
recordation of distance traveled since the beginning of each horizontal edge projection 
tracking sequence, and 

wherein horizontal edge projection tracks are sorted based on at least one of the 
following; 

^0 i) the duration of image inputs that the horizontal edge projection track records, 

m 

^ ii) the average length of the horizontal edge projection, and 

01 

s§ iii) whether the horizontal edge projection track at current image frame is in an 

updating mode, a non-updating mode, or is a new horizontal edge projection. 

0 

ffl 

ry 

P^15 6. The vision-based identification apparatus of claim 4 wherein the horizontal edge 

5 

j^, projection tracks are compared with predetermined parameters to determine if tracking 
possibilities exist; 

if tracking possibilities exist then the vision-based identification apparatus is utilized to 
track substantially horizontal edges through successive image inputs. 



20 



7. The vision-based identification apparatus of claim 6 wherein the predetermined 
parameters include at least one of the following; 
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i. the number of successive image inputs having horizontal edge projection 
tracks, and 

ii. the magnitude of the horizontal edge projection tracks. 

8. The vision-based identification apparatus of claim 4 wherein if no tracking 
possibilities exist, tracking may still occur for a pre-specified number of image inputs 
without tracking possibilities before the tracked horizontal edge projection track is 
discarded. 

9. The vision-based identification apparatus of claim 1 wherein tracking of horizontal 
edge projection vectors is assisted by using at least one of the following: 

i) vertical motion compensation; 

ii) forward motion compensation; 

wherein vertical motion compensation helps predict where tracked horizontal edge 
projection vectors will be located on successive image inputs by compensating for 
vertical motion, and 

wherein the forward motion compensation helps predict where tracked horizontal edge 
projection vectors will be located on successive image inputs by compensating for 
forward motion. 

10. The vision-based identification apparatus of claim 9 wherein forward motion 
compensation is achieved by using at least two previous points from the image input on 
the tracked horizontal edge projection tracks, immediately before the image input that 
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requires forward motion compensation for tracked horizontal edge projection vectors 
location prediction. 

1 1 . The vision-based identification apparatus of claim 9 wherein vertical motion is 
determined by: 

i. extracting vertical sHces of the image input; and 

ii. extracting one dimensional optical flow profiles of relative intensity of the 
vertical slices; and 

iii. averaging the optical flow of all vertical slices. 

12. The vision-based identification apparatus of claim 11 wherein the vertical slices are 
extracted fi*om each image input, and 

the relative extreme negative intensities relate to a change fi-om light to dark of an 
apparent horizon. 

13. The vision-based identification apparatus of claim 11 wherein the extreme negative 
intensities' average variation between rows on successive image inputs is an indicia of 
how an image collection device is moving vertically relative to at least one of the 
following: 

i. an apparent horizon; and 

ii. a distinct feature in the distance. 
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14. The vision-based identification apparatus of claim 1 wherein the signal input fi-om the 
image capture device to the image signal processor is a single horizontally centered 
window of the image signal input firom the image capture device. 

15. The vision-based identification apparatus of claim 14 wherein the single horizontally 
centered window is set at a predetermined width and a predetermined height and 
wherein the single horizontally centered window can be adjusted either left or right based 
on steering wheel position, or lane information. 

16. The vision-based identification apparatus of claim 14 wherein successive horizontal 
edge projection tracks have a length in excess of a predetermined length. 

17. The vision-based identification apparatus of claim 14 wherein if no tracking 
possibilities exist, a tracking protocol will be allowed to remain in operation for a pre- 
specified number of image inputs without tracking possibilities before the tracked 
horizontal edge projection track is discarded. 

18. A vision-based identification method comprising the steps of: 
providing a host vehicle; and 

equipping the host vehicle with a plurality of elements including: 

i) an image captiu-e device element; operatively interconnected with; 

ii) an image signal processor element; which is operatively interconnected with; 

iii) a matching processor element; which is also operatively interconnected with; 

Page 30 of 37 

Vision-Based Highway Overhead Structure Detection System 
HRL054/PD#000102 



iv) a radar transceiver element; 
wherein the image capture device is configured to provide a time based sequence of data 
frames to the image signal processor and the image signal processor provides a processed 
image signal to the matching processor; and 

wherein the data frames include a two dimensional array of pixel elements; and 
wherein the radar transceiver is configured to provide a radar signal to the matching 
processor; and 

wherein the matching processor combines the processed image signal and the radar 
signal, whereby the combined signals complement each other and allow for the effective 
identification of objects likely to be misidentified as collision threats. 

19. A vision-based identification method as set forth in claim 18, wherein the image 
capture device element is a video camera responsive to electromagnetic radiation 
substantially in at least one of the regions selected from the group consisting of the 
visible region, and the infrared region. 

20. A vision-based identification method as set forth in claim 18, wherein the time based 
sequence of data frames include a plurality of data elements including at least one 
horizontal edge; and 

wherein each horizontal edge is identified based on a plurality of pixels having a 
substantially similar electromagnetic radiation response across a plurahty of substantially 
horizontally aligned pixels; and 
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wherein the image signal processor extracts horizontal edges, from the time based 
sequence of data frames, in the form of edge pixels; and 

wherein the edge pixels are projected in each row of the data frames, to get a horizontal 
edge projection in the sequence of data frames; and 

wherein each horizontal edge projection may be tracked in time based on the horizontal 
edge projection's sequential appearance in the data frames. 

21 . A vision-based identification method as set forth in claim 20, wherein horizontal edge 
projections may be tracked in time by recording sequential data frames and matching 
horizontal edge projections in the sequential data frames while allowing, in the sequential 
data frames, for minor variations from, in at least one of the following categories: 

i. the relative position of the horizontal edge projection in the frame; 

ii. the relative orientation of the horizontal edge projection in the frame; and 

iii. the relative length of the horizontal edge projection; and 

wherein at least one new tracking sequence can exist for horizontal edge projections that 
have predefined characteristics but were not present in prior data frames. 

22. A vision-based identification method as set forth claim 21, wherein allowed, minor 
variations in the sequential data frames provide data that allows for the determination and 
recordation of distance traveled since the beginning of each horizontal edge projection 
tracking sequence, and 

wherein horizontal edge projection tracks are sorted based on at least one of the 
following; 
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i. the duration of image inputs that the horizontal edge projection track records; 

ii. the average length of the horizontal edge projection; and 

iii. whether the horizontal edge projection track at current image frame is in an 
updating mode, a non-updating mode, or is a new horizontal edge projection. 

23. A vision-based identification method as set forth in claim 21 wherein the horizontal 
edge projection tracks are compared with predetermined parameters to determine if 
tracking possibilities exist; 

if tracking possibilities exist then the vision-based identification apparatus is utilized to 
track substantially horizontal edges through successive image inputs. 

24. A vision-based identification method as set forth in claim 23 wherein the 
predetermined parameters include at least one of the following; 



iji i. the number of successive image inputs having horizontal edge projection tracks; 



pi 5 and 



^ ii. the magnitude of the horizontal edge projection tracks. 



25. A vision-based identification method as set forth in claim 21 wherein if no tracking 
possibilities exist, tracking may still occur for a pre-specified number of image inputs 
20 without tracking possibilities before the tracked horizontal edge projection track is 
discarded. 
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26. The vision-based identification method of claim 18 wherein tracking of horizontal 
edge projection vectors is assisted by using at least one of the following: 

i) vertical motion compensation; and 

ii) forward motion compensation; 

wherein vertical motion compensation helps predict where tracked horizontal edge 
projection vectors will be located on successive image inputs by compensating for 
vertical motion, and 

wherein the forward motion compensation helps predict where tracked horizontal edge 
projection vectors will be located on successive image inputs by compensating for 
forward motion. 

27. The vision-based identification method of claim 26, wherein forward motion 
compensation is achieved by using at least two previous points fi'om at the image input 
on the tracked horizontal edge projection tracks, immediately before the image input that 
requires forward motion compensation for tracked horizontal edge projection vectors 
location prediction. 

28. A vision-based identification method as set forth in claim 26, wherein vertical 
motion is determined by: 

i. extracting vertical slices of the image input; and 

ii. extracting one dimensional optical flow profiles of relative intensity of the 
vertical slices; and 

iii. averaging the optical flow of all vertical slices. 
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29. A vision-based identification method as set forth in claim 28, wherein the vertical 
slices are extracted fi-om each image input, and 

the relative extreme negative intensities relate to a change fi"om an electromagnetic 
radiation emission region having a greater intensity to an electromagnetic radiation 
emission region of lower intensity of an apparent horizon. 

30. A vision-based identification method as set forth in claim 28, wherein the extreme 
negative intensities' average variation between rows on successive image inputs is an 
indicia of how an image collection device is moving vertically relative to at least one of 
the following: 

i. an apparent horizon; and 

ii. a distinct feature in the distance. 

31. A vision-based identification method as set forth in claim 18, wherein the signal input 
fi-om the image capture device to the image signal processor is a single horizontally 
centered window of the image signal input firom the image capture device. 

32. A vision-based identification method as set forth in claim 31, wherein the single 
horizontally centered window is set at a predetermined width and a predetermined height; 
and 

wherein the single horizontally centered window can be adjusted either left or right based 
on steering wheel position, or lane information. 
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33. A vision-based identification method as set forth in claim 31, wherein successive 
horizontal edge projection tracks have a length in excess of a predetermined length. 

34. A vision-based identification method as set forth in claim 31 wherein if no tracking 
possibilities exist, a tracking protocol will be allowed to remain in operation for a pre- 
specified number of image inputs without tracking possibilities before the tracked 
horizontal edge projection track is discarded. 
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